Dynamic Programming – a Quick Review

Dynamic programming is a class of solution methods for solving sequential decision problems with a compositional cost structure. It is one of the major paradigms of algorithm design in computer science. The name was given in 1957 by Richard Bellman. The word “programming” both here and in linear programming refers to a tabular method that makes a series of choices, not to writing computer code. The word “dynamic” in this context conveys the idea that choices may depend on the current state, rather than being decided ahead of time. Typically, dynamic programming is applied to optimization problems. In such problems, there exist many possible solutions. Each solution has a value, and we wish to find a solution with the optimum value. There are two ingredients for an optimization problem to be suitable for a dynamicprogramming approach. One is that it satisfies the principle of optimality, i.e., each substructure is optimal. The other is that it has overlapping subproblems, otherwise a divide-and-conquer approach is the choice. The development of a dynamic-programming algorithm has three basic components: the recurrence relation (for defining the value of an optimal solution), the tabular computation (for computing the value of an optimal